The invention relates to computer generated graphics and performance capture, and more specifically to compression and streaming of optical performance data captured using a deformable mesh based representation.
Three-dimensional (3D) graphics are widely used in many fields such as animation movies, video games, virtual worlds, virtual reality, simulation, ergonomics, industrial design, architecture and many others. In 3D graphics, moving characters are typically represented using skeleton based representations. A skeleton based representation involves defining the articulating body parts of a 3D character, attaching a surface or skinning the skeleton, and defining how the skin moves as the skeleton moves. The skeleton is manually defined either using software or by placing markers on an actor and capturing the optical performance of the actor. However, more recently, approaches have developed for capturing optical performances with higher fidelity using a deformable mesh based representation of the optical performance. Deformable mesh based representations as well as other new surface based scene capture techniques offer a great level of detail during 3D graphics creation being able for example to capture motion of hair and clothes. Unfortunately, the resulting scene representation is less compact than skeleton based models, because the deformable meshes involve the use of large amounts of data to dynamically represent independent position data streams for every vertex. As such, in order to display each mesh in real time for computer graphics applications for example, specific compression methods may be necessary. The need for compression grows as the quality and detail of the meshes increase. Additionally, there is not yet a rich toolbox available which enables easy post-processing and modification of deformable mesh based 3D graphics creation.
Thus, there is a need to overcome these above noted challenges while also overcoming the obstacles and shortcomings in the art.